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DATA PROCESSING APPARATUS AND 



versions 



Field of Invention 

The present invention relates to encoding data pro|cesswig 
methods, which are arranged to introduce code words into 
In some applications the code words are used to uniquely i 

Correspondingly, the present invention also relates to 
and methods operable to detect one or more code words, whi 
material item. 

Background of the Invention 



identify 



• dsta 



A process in which information is embedded in mate 



identifying the material is known as watermarking. Watermarking can provide a 



METHOD 



apparatus and 
of material items, 
the material items, 
processing apparatus 
h may be present in a 



rial for the purpose of 



material. As such, if 
with the wishes of the 



3r used in a way, which 



facility for identifying a recipient of a particular version of th 
the material is copied or used in a way, which is inconsistent 

distributor of the material, the distributor can identify the material version from the 
identification code word and take appropriate action. 

In this description, an item of material, which is copied 
is inconsistent with the wishes of the originator, owner, creator or distributor of the 
material, will be referred to for convenience as an offending item of material^ or 
offending material. The material could be any of video, audio, audio/video material, 
software programs, digital documents or any type of information bearing material 

Any watermarking scheme should be arranged to mace it difficult for users 
receiving copies of the same material to collude to either mask 
or alter a watermark code word. A watermarking scheme sholuld therefore with high 
probability identify a marked material item, which has been the subject of a collusion 
attack. This is achieved by identifying a code word recove: 
material. Conversely, there should be a low probability of no 
when a code word is present (false negative probability). In addition the probability of 
falsely detecting a user as guilty of taking part in a collusion attack, when this user has 
not taken part, should be as low as possible (false positive prob ibility). 



ed from the offending 
detecting a code word 
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US Patent Serial No. 5, 664, 018 discloses a watermarking process in which a 
plurality of copies of material items are marked with a digital watermark formed from 
a code word having a predetermined number of coefficients. The watermarked 
material item is for example an image. The apparatus for introducing the watermark 
transforms the image into the Discrete Cosine Transform (DCT) domain. The digital 
watermark is formed from a set of randomly distributed coefficients having a normal 
distribution. In the DCT domain each code word coefficient is added to a 
corresponding one of the DCT coefficients. The watermarked image is formed by 
performing an inverse DCT. A related publication entitled "Resistance of Digital 
Watermarks to Collusion Attacks", by J. Kilian, F. T. Leighton et al, published by 
MITJ July 27, 1998, provides a detailed mathematical analysis of this watermarking 
process to prove its resistance to attack. Co-pending UK patent applications 
0129865.2, 0129841.3, 0129840.5, 0129907.2 and 0129836.3 disclose encoding and 
detecting apparatus and methods which aim to provide a practical watermarking 
systen, which utilises code words having coefficients which are randomly distributed 
as proposed in US 5,664,018. 
Summary of Invention 



According to the present invention there is provided a detecting data processing 
appailatus operable to determine whether one or more code words of a predetermined 
set of code words is present in a suspected version of a material item. The apparatus 
comprises a registration processor operable to associate samples of the suspected 
version with samples of a copy of the original material item, a recovery processor and 
a detection processor. The recovery processor is operable to generate a recovered code 
word by comparing the registered copy of the original and the suspect material items. 
The detection processor is operable to detect one or more code words from a 
correlation between the recovered code word and the code words from the set. The 
regist-ation processor is operable to form at least one reduced-bandwidth- version of 
the material item and to associate the suspected version and the copy of the material 
item in accordance with a comparison between the reduced-bandwidth-versions of the 
suspected version and the original material item. 
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The present invention aims to provide an improver lent in the registration 
between the suspected version and the original version of a material item. The 
registration of the suspected version and the original version of a material item is 
typically required in order to correctly recover a code word from the suspected 
version. As explained in co-pending UK patent application 0129907.2, the code word 
is recovered from the suspected material item by comparing respective samples of the 
suspected version with the samples of the original materiii item. However, by 
providing a registration processor to associate the samples of the suspected version and 
the original version of the material items, the code word w: 11 be more likely to be 
correctly recovered. Accordingly, the likelihood of correctly detecting the code word 
will be improved, reducing the false negative probability. For material items such as 



of samples may need to 
a result, the registration 
difficult. However, by 
and the original version 



video recordings of motion picture films, a substantial number 
be associated in order to effect the registration process. As 
process may be time consuming or at least computationally 

1 5 forming reduced-bandwidth-versions of the suspected version 

of the material items, a corresponding reduction in the tide taken to perform the 
registration process is provide in accordance with the bandwid h reduction. 

The registration processor may be operable in some embodiments to generate a 
plurality of reduced-bandwidth-versions of the suspected and 

20 material items in accordance with a plurality of levels. Each 
increasing reduction in the bandwidth of the suspected and 
Advantageously, the association of the suspected version and 
version may be determined in accordance with a nested alignment process. The nested 
alignment process is provided by comparing reduced-bar dwidth- versions of the 

25 suspected and original material items on a lower bandwidth 

misalignment and making an adjustment to the reduced biindwidth suspected and 
original material items at the next lowest level according to the determined 
misalignment and so on. As a result a time to perform the re jistration process can be 
reduced, whilst the accuracy of the registration process is increased, as a result of the 

30 progressively increased resolution of the material items being 
In preferred embodiments, the suspected version is 
formed by combining a code word with part of the bandwidth 



the copy of the original 
level corresponds to an 
original material items, 
the copy of the original 



compared, 
assumed to have been 
of the original material 
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item. The detecting data processing apparatus includes a bandwidth adaptation 
processor operable in combination with the recovery processor to form reduced- 
bandwidth-versions of the suspected version and the original versions of the material 
items, or a difference between the suspected and original material items. The 
registration processor is operable in combination with the bandwidth adaptation 
processor to perform the association as part of the bandwidth reduction. As such, 
although the reduced-bandwidth-versions of the suspected antianginal material items 
or their difference is required to recover the code word the registration process is 
integrated with the bandwidth reduction. For example, the code word may have been 
introduced into at least one of the temporal or spatial low bandwidth of the suspect 
mateiial item and correspondingly the bandwidth adaptation processor is operable to 
perfo m the bandwidth reduction at least one of temporally or spatially. The plurality 
of recjuced-bandwidth-versions may therefore be formed temporally or spatially. 

Various further aspects and features of the present invention are defined in the 
appeijded claims. 
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lata processor according 



Brief Description of Drawings 

Embodiments of the present invention will now Ik described by way of 
example only with reference to the accompanying drawings, where like parts are 
provided with corresponding reference numerals, and in which: 
5 Figure 1 is a schematic block diagram of a data processing apparatus which 

operates as a watermark encoder; 

Figure 2 is a schematic block diagram of a detecting dalta processing apparatus; 

Figure 3A is a representation of an original inage, Figure 3B is a 
representation of a marked image and Figure 3C is the marked 
10 Figure 4 is a graphical representation of an example correlation result for each 

of a set of N code words; 

Figure 5 is a schematic block diagram of an encoding 
to an embodiment of the invention; 

Figure 6 is a schematic block diagram of a temporal up-sampler forming part of 
1 5 the encoding data processor shown in Figure 5; 

Figure 7 is a schematic block diagram of a spatial up- 
the encoding data processor shown in Figure 5; 

Figure 8 is a schematic block diagram of a spatial sub - 
the data processing apparatus shown in Figure 5; 
20 Figure 9 is a schematic block diagram of a temporal sub-sampler forming part 

of the data processing apparatus shown in Figure 5; 

Figure 10 is a schematic block diagram of a detecting data processing 
apparatus, which operates to detect watermark code words in the marked material 
items; 

25 Figure 1 1 is a representation of spatially sub-sampl ^d video frames having 

three levels of sub-sampling to illustrate a nested alignment process according to an 
embodiment of the invention; and 

Figure 12 is a flow diagram representing the operation of the registration 
processor according to the nested alignment process embodyir g the invention. 



sampler forming part of 
sampler forming part of 
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Description of Preferred Embodiments 



Watermarking System Overview 



An explanation of a watermarking technique utilised by embodiments of the 
present invention will now be described with reference to protecting, for example 
video images. The number of users to which the video images are to be distributed 
determines the number of copies. A watermark code word is added to each copy 
which identifies the copy assigned to one of the users. 

Video images are one example of material, which can be protected by 
embec ding a digital code word. Other examples of material, which can be protected 
by embedding a code word, include software programs, digital documents, music, 
audio signals, audio/visual material, multi-media content and any other information- 
bearing signal. 

An example of an encoding image processing apparatus, which is arranged to 
introduce an identification code word into a copy of an original image, is shown in 
Figure 1 . An original image / is received from a source and stored in a frame store 1 . 
This original image is to be reproduced as a plurality of water marked copies, each of 
which is marked with a uniquely identifiable code word. The original image is passed 
to a D iscrete Cosine Transform (DCT) processor 2, which divides the image into 8x8 
pixel blocks and forms a DCT of each of the 8x8 pixel blocks. The DCT processor 2 
therefore forms a DCT transformed image V. 

In the following description the term "samples" will be used to refer to discrete 
samples from which an image (or indeed any other type of material) is comprised. The 
samples may be luminance samples of the image, which are otherwise, produced from 
the imige pixels. Therefore, where appropriate the terms samples and pixels are inter- 
changeable. 

The DCT image Fis fed to an encoding processor 4. The encoding processor 4 
also receives identification code words from an identification code word generator 8, 

The code word generator 8 is provided with a plurality of seeds, each seed 
being used to generate one of the corresponding code words. Each of the generated 
code v/ords may be embedded in a copy of the original image to form a watermarked 
image, The code word generator 8 is provided with a pseudo random number 
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generator. The pseudo random number generator produces the code word coefficients 
to form a particular code word. In preferred embodiments the coefficients of the code 
words are generated in accordance with a normal distiibution. However, the 
coefficients of the code word are otherwise predetermined in accordance with the seed, 
which is used to initialise the random number generator. Thus, for each code word 
there is a corresponding seed which is stored in a data store 12. Therefore, it will be 
understood that to generate the code word A 7 , seedj is retrieved from memory 12 and 



word generator 8. 
the original image is 



other embodiments the 
in the spatial domain or 

code word coefficients, 



used to initialise the random number generator within the code 

In the following description the DCT version of 
10 represented as V, where; 

K = { v /} = { v i>v 2 ,v 3 ,v 4 , v„} 

and v/ are the DCT coefficients of the image. In 

samples of the image v,- could represent samples of the image 

in an alternative domain. 

1 5 Each of the code words X* comprises a plurality of n 

where; 

X 1 ~ {xj}~ ^c[ 9 x' 29 x l Z9 x f 49 x l n } 

The number of code word coefficients n corresponds tb the number or samples 
of the original image V! However, a different number of coefficients is possible, and 
20 will be determined in dependence upon a particular applicatior 

A vector of code word coefficients X* forming the z-th code word is then passed 
via channel 14 to the encoder 4. The encoder 4 is arranged 
image W\ by adding the code word X* to the image V. Effectively, therefore, as 
represented in the equation below, each of the code word coefficients is added to a 
25 different one of the coefficients of the image to form the wateijnark image Wj 

W'^V + X* 

W § = v, +x[ 9 v 2 + xj,v 3 +x£,v 4 +*J,....: ,v„ +£ 
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As shown in Figure 1, the watermarked images W\ are formed at the output of 
the irhage processing apparatus by forming inverse DCT of the image produced at the 
outpi t of the encoding processor 4 by the inverse DCT processor 18. 

Therefore as represented in Figure 1 at the output of the encoder 4 a set of the 
wateijmarked images can be produced. For a data word of up to 20-bits, one of 1 0 000 
000 code words can be selected to generate 1 0 million watermarked W[ versions of the 
original image /. 

As shown in Figure 1 , the DCT processor 2, the encoding processor 4 and the 
inverfce DCT processor 18 form a watermark encoder 120. The watermark encoder 
120 dso forms part of a data processing apparatus shown in Figure 5, which will be 
explained shortly. Furthermore the data store 12 and the code word generator 8 form a 
watermark generator 1 16, which also appears in Figure 5. 

Although the code word provides the facility for uniquely identifying a marked 
copy W\ of the image /, in other embodiments the 20 bits can provide a facility for 

communicating data within the image. As will be appreciated therefore, the 20 bits 
used to select the identification code word can provide a 20 bit pay-load for 
communicating data within the image V. 

The encoding image processing apparatus which is arranged to produce the 
wateilmarked images shown in Figure 1 may be incorporated into a variety of products 
for different scenarios in which embodiments of the present invention find application. 
Generally the distribution of the watermarked images produced by the encoding image 
processing apparatus shown in Figure 1 is represented by a distribution cloud 19. 
Detei tins Processor 



A detecting image processing apparatus which is arranged to detect one or 
more of the code words, which may be present in an offending marked image is shown 
in Figure 2. Generally, the image processor shown in Figure 2 operates to identify one 
or more of the code words, which may be present in an offending copy of the image. 

The offending version of the watermarked image W is received from a source 
and stored in a frame store 20. Also stored in the frame store 24 is the original version 
of the image /, since the detection process performed by the image processor requires 
the original version of the image. The offending watermarked image W and the 
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the watermarked image 
the identification code 



original version of the image are then fed via respective connecting channels 26, 28 to 
a registration processor 30. 

As already explained, a suspect version of the imajge W may have been 
produced by photographing or otherwise reproducing a part of 
W. As such, in order to improve the likelihood of detecting 
word, the registration processor 30 is arranged to substantially align the offending 
image with the original version of the image present in the da :a stores 20 and 24. The 
purpose of this alignment is to provide a correspondence between the original image 
samples / and the corresponding samples of the watermarked 
code word coefficients have been added. 

The effects of the registration are illustrated in Fig 
example of the original image / is shown with respect to an o 
of the image W\ As illustrated in Figure 3, the watermarked 
respect to the original image / and this may be due to the relative aspect view of the 
camera from which the suspect version of the watermarked imnge was produced. 

In order to recover a representation of the code word 
samples of the original image should be subtracted from the corresponding samples of 
the marked offending image. To this end, the two images ar i aligned. As shown in 
Figure 3, the registered image PP'has a peripheral area PA wHch includes parts which 
were not present in the original image. 

As will be appreciated in other embodiments, the regis 
not be used because the offending image W may be already 



image W to which the 

jre 3. In Figure 3 an 
fending marked version 
image W is offset with 



ration processor 30 may 
substantially aligned to 



the originally version of the image /, such as, for example, if the offending version was 
downloaded via the Internet. Accordingly, the detecting ima^e processor is provided 
with an alternative channel 32, which communicates the marked image directly to the 
recovery processor 40. 

The registered image W" is received by a recovery processor 40. The recovery 
processor 40 also receives a copy of the original image / via £ second channel 44. The 
registered image fP'and the original image / are transformed by a DCT transform 



processor 46 into the DCT domain. An estimated code wo *« 



d X f is then formed by 



POM 384GB 10 

subtracting the samples of the DCT domain marked image F'from the DCT domain 
samples of the original image Vas expressed by the following equations: 
X'= V- V 

= vf-v„v£ -v 2 ,v; -vj.vj -v 4 , A-v n , 

r t / / t 

— Xj , x 2 , x 3 , x 4 , X n 

The output of the recovery processor 40 therefore provides, on a connecting 
channel 50, an estimate of the coefficients of the code word which is to be identified. 
The recovered code word X' is then fed to a first input of a correlator 52. The 

correlator 52 also receives on a second input the regenerated code words X 1 produced 
by ths code word generator 54. The code word generator 54 operates in the same way 
as the code word generator 8 which produces all possible code words of the set, using 
the predetermined seeds which identify uniquely the code words from a store 58. 

The correlator 52 forms n similarity sim(i) values. In one embodiment, the 
simil irity value is produced by forming a correlation in accordance with following 
equation: 

. , , x X* ' X' x * x[ +■ x, ■ x\ + Xj • x-i + + x ' x 

sim(i) = . = / 

4x l -r Tjx l i -x{ + x 2 i -x' 2 + x, i .x' 3 + + xj -x' 

Each of the n similarity values sim(i) is then fed to a detector 60. The detector 
60 then analyses the similarity values sim(i) produced for each of the n possible code 
words. As an example, the similarity values produced by the correlator 52 are shown 
in Figure 4 with respect to a threshold TH for each of the possible code words. As 
shown in Figure 4, two code words are above the threshold, 2001, 12345. As such, the 
detecting processor concludes that the watermarked version associated with code word 
2001 and code word 12345 must have colluded in order to form the offending image. 
Therefore, in accordance with a false positive detection probability, determined from 
the population size, which in this case is 10 million, and the watermarking strength a, 
the height of the threshold TH can be set in order to guarantee the false detection 
probability. As in the example in Figure 4, if the similarity values produced by the 
corre ator 52 exceed the threshold then, with this false p03itive probability, the 
recipi ents of the marked image are considered to have colluded to form the offending 

watermarked version of the image 
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Bandwidth Adaptive Encoding Data Processor 



An improved encoding data processor according to 



invention is shown in Figure 5. As will be explained shortly, the encoding data 



processor is arranged to introduce a code word into a pari 



an embodiment of the 



of the bandwidth of a 



comprises a bandwidth 
and a second adaptation 



5 material item. In the example embodiment shown in Figure 5, the code word is 
introduced into a low bandwidth part of a material item such as for example a motion 
picture film or video recording. As will be explained, this is effected by assuming that 
the code word corresponds to a low bandwidth part of the material. The bandwidth of 
the code word is then up-converted to the bandwidth of the material and combined 
1 0 with a copy of the material to form a watermarked version of the material. However in 
other embodiments the code word may be combined with) another portion of the 
' material item, bandwidth. 

The encoding data processor shown in Figure 5 
processor 100 formed from a first adaptation processor 102 
15 processor 104. A watermark code word generator 106 generates code words, which 
are fed via a code word processor 108 to the first adapter 102. An original material 
item is fed from a source 1 10 to the second bandwidth adaptation processor 104. 

The first adaptation processor 102 is operable to increase the bandwidth of the 
code word to the bandwidth of the material item. The second adaptation processor J 04 
20 is operable to form a reduced-bandwidth-version of the origin il material item, which is 
fed to a registration processor 1 12. The registration processoi is operable to associated 
the samples of the reduced-bandwidth-version of the Material item with the 
coefficients of the code word which are to be added to the Material item. In order to 
reduce a likelihood of the code word being perceivable in ti e material item after the 
code word has been combined with the material item, the code word coefficients are 
adapted in accordance with a sensitivity of the material to c irry the coefficients. To 
this end, the samples to which the code word coefficients 
watermark processor 108, are made available to the watermar c 



are to be added by the 
processor 108, from the 
registration processor 1 12 via a connecting channel 1 14. Te chniques for adapting the 

the material to carry the 



30 code word coefficients in accordance with a sensitivity of 



coefficients whilst maintaining a predetermined false positive and false negative 
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detection probability are detailed in co-pending UK patent applications number 
0129865.2. 

The increased bandwidth version of he code word is fed to a first input 1 16 of a 
combining processor 118. To a second input 120 of the combining processor 118, a 
copy of the original material item is fed. The combining processor is arranged to 
comt ine the increased bandwidth version of the code word with the copy of the 
material item to form a watermarked version 122 of the material item at an output 124. 
Therefore, in combination, than bandwidth adaptation processor 100 and the 
combining processor 118 combine the code word with the low frequency components 
of the material item. As will be appreciated in other embodiments the code word may 
be combined in another portion of the bandwidth of the material item. 

The bandwidth reduction performed by the second adaptation processor 104 
should correspond to the bandwidth increase applied by the first adaptation processor 
102. As a result, the reduced-bandwidth-version of the material item should have a 
bandwidth which is substantially the same as the watermark code word. As will be 
appreciated, by matching the bandwidth reduction applied to the material item to the 
bandwidth of the code word, the samples of the reduced-bandwidth-version of the 
material item and the code word coefficients can be associated by the registration 
proce ssor 112. Accordingly, the watermark processor 108 can adapt the coefficients 
of the code word in accordance with the sensitivity of the samples to carry the code 
word. However some embodiments may not include the second adaptation processor 
104, the registration processor 112 or the watermark processor 108. Such 
embodiments would only require the first adaptation processor of the bandwidth 
processor 100 and the combining processor 118. 

As explained above, the first adaptation processor 102 is arranged to increase 
the bandwidth of the watermark code word, so the that increased bandwidth 
corresponds to the bandwidth or part of the bandwidth of the material item. There are 
variois techniques for effecting this bandwidth increase. For the example embodiment 
showr in Figure 5, the bandwidth increase is effected in both the temporal and spatial 
domains. In other embodiments, the bandwidth increase may be spatially or 
temporally effected. The first adaptation processor comprises a temporal up-sampler 
126 and a spatial up-sampler 128. An example implementation of a temporal up- 
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rate increase processor 
by introducing samples 



-sampled version of the 
e code word samples is 



sampler 126 is shown in Figure 6 and an example of a spatial pp-sampler 128 is shown 
in Figure 7. 

The temporal up-sampler 126 shown in Figure 6 domprises a sample rate 
increase processor 140 and a low pass filter 142. The sampl e 
140 is arranged to. increase the sample rate of the code wore 
between the code word coefficients in accordance with a desired temporal bandwidth 
increase. The low-pass filter has a pass-band, which corresponds to the temporal 
bandwidth of the material item. As such by passing the up 
code word through the low-pass filter, an interpolation of tl 
formed, which has an effect of spreading the temporal banc width of the code word 
across the bandwidth of the material item. 

The spatial up-sampler shown in Figure 7 comprises a wavelet forming 
processor and an inverse wavelet transformer 146. The up-sampler performs an eight 
times spatial up-sampling by forming the code word into the lowest frequencies sub- 
band of a second order wavelet transform. The second order wavelet transform is 
formed by the wavelet forming processor 144. The coefficier ts of the other sub-bands 
are set to zero. The inverse wavelet transformer then performs an inverse wavelet 
transform on the formed wavelet, which has an effect of spatially up-sampling the 
code word by a factor eight. 

Returning to Figure 5, the second adaptation processor which produces the 
reduced-bandwidth- version of the material item comprises a spatial sub-sampler 132 
and a temporal sub-sampler 134. As will be appreciated, there are various techniques 
for implementing a bandwidth reduction on a data signal. However as explained 
above the second adaptation processor generates the reduced- band width- version of the 
original material item in correspondence with the bandwidth increase performed on the 
code word. Figures 8 and 9 provide block diagrams of exanple implementations of 
the spatial and temporal sub-samplers 132, 134. 

In Figure 8, the spatial sub-sampler comprises a wavelet transformer 204 and a 
wavelet selector 206. The spatial sub-sampling is implemented using the wavelet 
transform. To this end, the wavelet transformer 204 is arranged to generate a second 
order wavelet transform of the material. The wavelet selector 206 is then arranged to 
select the lowest frequency wavelet sub-band, setting the wavelet coefficients in the 
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other 



wavelet transform of a video frame 210, formed by the wavelet transformer 204, and 



an il 



sub-bands to zero. This is illustrated in Figure 8, by a representation of a 



ustration of the wavelet transformed frame 212 formed after processing by the 



wavelet selector 206. By selecting the lowest frequency wavelet sub-band of a second 
ordei wavelet transformed image, a spatially sub-sampled version of the video frame is 
form id, which has been down-sampled by a factor of eight. 

Correspondingly, the temporal sub-sampler 134 illustrated in Figure 8 includes 
a lov[-pass filter 220, which functions as an anti-aliasing filter for the temporally sub- 
sampled material. The bandwidth of the low pass filter is therefore set in accordance 
with a rate at which the material is to be sub-sampled. For the illustrative example, the 
temp3ral sub-sampling rate is four. Accordingly, after the low pass filter 220, a frame 
selector 222 is operable to select every fourth frame of the material, discarding the 
other frames, to form the temporally sub-sampled material. 
Detection Data Processor 



As already explained, a code word which is present in a marked version of a 
material item, can be detected with a predetermined false positive and false negative 
detection probabilities. The detection data processor illustrated in Figure 2 can be 
arranged correspondingly to detect whether a code word is present in a suspect 
material item. However, because the code word has been combined with a low 
bandwidth component of the material item, some adaptation of the detecting data 
processor shown in Figure 2 is required. A data detecting processor according to an 
example embodiment of the invention is shown in Figure 10. 

In Figure 10 the suspect material is fed from a store 400 to a spatial bandwidth 
processor 404. Also fed to the spatial bandwidth processor 404 is a copy of the 
origir.al source material from a second store 402. The spatial bandwidth processor 404 
comprises a first spatial sub-sampler 406 and a second spatial sub-sampler 408. The 
first md second spatial sub-samplers 404, 406 are arranged to perform spatial sub- 
samp ing at a rate which corresponds to the up-sampling performed by the spatial up- 
samp: er 128 performed by the encoding data processor illustrated in Figure 5. For this 
examole, the spatial sub-sampling rate is one eighth. The spatial sub-samplers 406, 
408 can be implemented as illustrated in Figure 7, as explained above. As will be 
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appreciated the same spatial sub-sampler could be used for Ipoth the suspect and the 
original material versions. 

The spatially sub-sampled versions of the suspect and the original material 
items are receive by a temporal registration processor 410 from connecting channels 
412, 414. The temporal registration processor 410 operates 
suspect and the original material items to the effect that the 
version are correspondingly synchronised to the samples of the suspect version of the 
material item 

The temporally aligned suspect and original material 
temporal bandwidth processor 416. The temporal bandwidth 



first temporal sub-sampler 418 and a second temporal sub-sampler 420, which are 



arranged to received the temporally aligned suspect and 



to temporally align the 
samples of the original 



items are then fed to a 
orocessor 416 includes a 



original material items 



respectively. Temporal sub-sampling is performed by the first and second temporal 
sub-samplers 418, 420 on the suspect and original material [terns in correspondence 
with the temporal up-sampling rate performed by the up-samnler 126 in the encoding 
data processing apparatus 101 of Figure 5. The temporally a id spatially sub-sampled 
version of the suspect and original material items are thei 
registration processor 422. The spatial registration processor 422 is operable to 
provide a spatial alignment of the samples of the suspect and the original material 
items. 

The spatial and the temporal bandwidth processors 4j04, 416 operate to form 
reduced-bandwidth-versions of the suspect and copy of the original material items. In 
effect the temporal and spatial registration processors 413, 422 are arranged to 
associate the samples of the reduced-bandwidth-versions of ihe suspect and the copy 
of the original material items. Effectively, therefore a subtracting processor 40' 
receives respective samples of the suspect and the original material items to which 
code word coefficients may have been added. A result of the subtraction is to form a 
vector of samples X which represents a recovered code word. The recovered code 
word X'\s then fed to a correlator 52' which operates in combination with the detector 
60', as already explained with respect to the detecting data processor of Figure 2 to 
determine whether the recovered code word corresponds tc one of the set of code 
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code 
linear 



words. The explanation will, therefore not be repeated here. The detected code word 
can then uniquely identify a recipient of the offending material item. 

The operation of the encoding data processor to adapt the bandwidth of the 
! [word and combine the code word coefficients with the material item represents a 
■ mathematical process. As a result in alternative embodiments, the detecting data 
processor may be arranged to subtract the copy of the material item from the suspect 
material item and then proceed to form a reduced-band width- version of the difference. 
The educed-band width-version of the difference is then correlated with the code 
word > from the set of code words to identify if a code word is present in the suspect 
mateiial as already explained. However, as will be explained in the following 
paragraphs, forming the reduced-bandwidth- versions of the suspect and the original 
mateiial items provides an advantage in reducing an amount of processing required to 
register the suspect and copy of the original material items. 
Improved Registration Process 



copy 



from 
of the) 



differs 



Embodiments of the present invention can provide an improved facility for 
aligning the spatially and temporally sub-sampled versions of the suspect and the copy 
of thi original material item. As such an improvement can be provided in the 
probability of correctly detecting a code word which is present in an offending 
material item. The improvement in the registration of the suspect material with the 
:>f the original material is provided by a multi-stage nested alignment process. 
As indicated above, the code word detection processor includes a registration 
processor comprising the temporal alignment processor 410 and the spatial alignment 
processor 422. The temporal alignment processor 410 is operable to provide a first 
temporal alignment of the suspect and original material items. There are various ways 
in which this can be achieved. For example, for material representing moving images 
such as a film or a video reproduction, scene changes in the material can be detected 
significant changes in the chrominance and/or luminance values between frames 
material. By matching the relative times at which these changes occur, in the 
suspett and original material items, an estimate of temporal alignment is provided. 
However as will be appreciated, the suspect material may have a frame rate which 
from the original because the suspect material has been generated in accordance 
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sust ect 



with a different recording or storage format. As such, in 
change which falls between a frame boundary, the suspect 
sampled by introducing a frames amongst those of the suspe 
with an increased frame rate. Low-pass filtering the. up 
provides an interpolated version of the suspect material from 
be more easily identified. 

An advantage is provided by performing temporal al 
and original material items have been sub-sampled. This is b 
been embedded in the low bandwidth portion to form the 
this reason, the code word in the material will be less like 
sampling the suspect and original material items before 
the spatial bandwidth processor 404 can sub-sample the 
items before temporal alignment is performed by the temporjal 
406, 408. As a result a time taken to determine the tempora 
be substantially reduced, as a result of a reduction in an amouit 

As explained above, after the temporal sub-san 
bandwidth processor 416, the spatial registration processor 
alignment between the suspect and the original material iterfis 
spatial registration processor 422 will now be explained in 
with reference to Figure' 1 1 . 

The spatial alignment processor 422 is arranged to 
spatially sub-sampled versions of the suspect and original 
example illustrated in Figure 1 1 , two increasingly spatially 
formed SSV1, SSV2. For a first example frame C of the 
increasingly sub-sampled versions SSV1, SSV2 produce two 
frame C, of the sub-sampled suspect material version 
increasingly sub-sampled versions A', B' for the two levels 
for the original material item frame C The spatial alignment 
operable to perform a nested spatial alignment process as follows 

(i) The most spatially sub-sampled frames A, A' 
material items are compared; 



order to identify a scene 
material may be further up- 
t material in accordance 
sjampled suspect material 
which a scene change can 

gftment after the suspect 
bcause the code word has 
ma|rked material items. For 
y to be effected by sub- 
temporal alignment. Hence 
and original material 
registration processors 
alignment estimate may 
of processing required, 
pling by the temporal 
performs a further spatial 
The operation of the 
the following paragraphs 

fprm a set of increasingly 
material items. For the 
-sampled versions are 
suspect material item, two 
frames A, B, from a first 
The correspondingly 
SSV1, SSV2 are formed 
processor 422 is then 



:>ub- 



cf 



the suspect and original 
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(ii) A result of the comparison provides an indication of the spatial 
misalignment between the suspect and original material versions for the most spatially 
sub-sampled frame A, A'; 

(iii) In accordance with any misalignment of the frames A, A', determined 
in (ii) an alignment of the next most spatially sub-sampled frames B, B' is performed; 

(iv) The next most spatially sub-sampled frames B, B' of the suspect and 
original material items are then compared; * 

(v) A result of the comparison provides an indication of any spatial 
misalignment between the suspect and original material versions for the least spatially 
sub-sampled frames C, C; 

(vi) In accordance with the determined misalignment of the frames B, B' of 
the bvel SSV1, an alignment of the least spatially sub-sampled frames C, C is 
performed; 

(vii) Steps (i) to (vi) are repeated for a subsequent set of increasingly sub- 
sampled frames. 

As illustrated in Figure 11, the nested alignment process for increasing levels 
of sp itially sub-sampled frames may not be performed for every frame of the suspect 
and original material items. The nested alignment of frames may be performed at a 
rate vhich corresponds to an amount by which spatial misalignment between the 
suspect and original material may have occurred. Therefore as illustrated in Figure 1 1, 
neste i alignment is performed every other frame. It will be appreciated that other rates 
are envisaged in accordance with a possible rate of change of misalignment such as 
one in three or one in four frames are aligned in accordance with the nested process. 

The nested alignment process for different amounts of spatial sub-sampling has 
an advantage in both improving the accuracy of the spatial alignment, and reducing an 
amount of processing which is required to effect spatial alignment. In addition, the 
application of the code word to the low spatial frequencies reduces the sensitivity of 
the code word detection process to misalignment. 

A flow diagram illustrating the operation of the registration processor is 
provided in Figure 12. The operation of the registration processor is summarised as 
follows with reference to Figure 12: 
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S.l: The spatially sub-sampled suspect and original rraterial items are received 
by the temporal alignment processor 410. 

S.2: Transitions, scene changes or edit points in the [naterial are detected and 
temporal alignment is performed. 

S.3: The spatially sub-sampled suspect and original material items are 
temporally sub-sampled. 

S.4: For a sample frame of the temporally and spatikMy sub-sampled suspect 
and original material items, two further increasingly spatially sub-sampled frames are 
formed corresponding to two increasingly spatially sub-sampled levels SSV1, SSV2 

S.5: For the lowest level SSV2, comprising the most temporally and spatially 
sub-sampled frames A, A', the frames A, A' are compared and any misalignment 
determined. 

S.6: The frames at the next level of sub-sampliilg SSV1 are aligned in 
accordance with the determined misalignment for the most su>sampled frames A, A'. 

S.7: The next most sub-sampled frames B, B' 
misalignment determined. 

S.8: In accordance with the misalignment determinb 
sampled frames B, B', the least sub-sampled frames of the suspect and original 
material items C, C are aligned. 

The frame alignment process is then repeated for subsequent frames. 

Various further aspects and features of the present invention are defined in the 
appended claims. Various modifications can be made to 



are compared and any 



d for the next most sub- 



the embodiments herein 



before described without departing from the scope of the present invention. 
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CLAIMS 

1. A detecting data processing apparatus operable to determine whether 
one olr more code words of a predetermined set of code words is present in a suspected 
version of a material item, the apparatus comprising 

a registration processor operable to associate samples of the suspected version 
with ^amples of a copy of the original material item, 

a recovery processor operable to generate a recovered code word by comparing 
the registered copy of the original and the suspect material items, and 

a detection processor operable to detect one or more code words from a 
correlation between the recovered code word and the code words from the set, 
wherein the registration processor is operable to fonn at least one reduced-bandwidth- 
version of the material item and to associate the suspected version and the copy of the 
material item in accordance with a comparison between the reduced-bandwidth- 
versions of the suspected version and the original material item. 

2. A detecting data processing apparatus as claimed in Claim 1, wherein 
the registration processor is operable to generate a plurality of reduced-bandwidth- 
versions of the suspected and of the copy of the original material items in accordance 
with a plurality of levels, each level corresponding to an increasing reduction in the 
bandwidth of the suspected and of the original material items, the association of the 
suspected version and of the copy of the original version being determined in 
accordance with a nested alignment process. 

3. A detecting data processing apparatus as claimed in Claim 1, wherein 
the registration processor is operable to associate the suspected version and the 
original version according to the nested alignment process by 

comparing reduced-bandwidth-versions of the suspected and of the original 
material items according to a lowest level having the smallest bandwidth, 

determining any misalignment between the reduced-bandwidth-versions of the 
suspejcted and of the original material items according to the lowest level, 
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V 



aligning the reduced-bandwidth-versions of the suspected and of the original 
material items according to next lowest level, and 

repeating for the next lowest level until the suspected 
version of the material item are aligned. 



spec ted 



4. A detecting data processing apparatus as claimed 
Claim, wherein the reduced-bandwidth-versions of the sus; 
of the original material item are formed by a bandwidth 
bandwidth reduction being formed in at least one of the 
domains. 



version and the original 



in any preceding 
version and the copy 
adaptation processor, the 
spatial or the temporal 



5. A detecting data processing apparatus as cliimed in any preceding 
Claim, wherein the suspected version is assumed to have been formed by combining a 
code word with part of the bandwidth of the original materia item, the detecting data 
processing apparatus including a bandwidth adaptation processor operable in 
combination with the recovery processor to form reduced-bciid width- versions of the 
suspected version or of the original versions of the material items, or of a difference 
between the suspected and the original material items, the registration processor being 
operable in combination with the bandwidth adaptation processor to perform the 
association as part of the bandwidth reduction. 

6. A detecting data processing apparatus as clained in Claim 5, wherein 
the code word has been introduced into at least one of the temporal or spatial 
bandwidth of the suspect material item and correspondingly the bandwidth adaptation 
processor is operable to perform the bandwidth reduction at least one of temporally or 
spatially. 



7. A detecting data processing apparatus as 
wherein the bandwidth adaptation processor is operable to 
versions of the suspected and of the original material items 
the suspected and the original versions of the material, 
comprising a temporal alignment processor and a spatial 



claimed in Claim 5 or 6, 
brm reduced bandwidth 
hy spatially sub-sampling 
the registration processor 
Alignment processor, the 
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temporal alignment processor being operable to temporally align the spatially reduced- 
band width- version of the suspected and of the original copy of the material item, and 
the spatial alignment processor is operable to perform the nested alignment process, 
the plurality of reduced-bandwidth-versions being formed spatially for the nested 
alignment process. 

8. A detecting data processing apparatus as claimed in any preceding 
Claiik wherein the correlation processor includes a code word generator operable to 
gene] ate pseudo-random numbers from which the regenerated code word coefficients 
are formed, the pseudo-random numbers being generated from a seed value uniquely 
associated with the code word. 

9. A detecting data processing apparatus as claimed in Claim 8, wherein 
the s£ed value is formed from the samples of the marked material item. 

10. A detecting data processing apparatus as claimed in any preceding 
Claiih, wherein the code word has been introduced into the material item in the 
discrete cosine transform domain, the apparatus comprising 

a discrete cosine transform processor operable to transform the suspected 
reducjed-bandwidth-version of the material item and the reduced-bandwidth-copy of 
the criginal material item into the discrete cosine transform domain, wherein the 
recovery processor is operable to generate the recovered code word by subtracting 
corresponding discrete cosine transform coefficients of the original material version 
from discrete cosine transform coefficients of the marked material version. 

11. A method of determining whether one or more code words of a 
predetermined set of code words is present in a suspected version of a material item, 
the apparatus comprising 

associating samples of the suspected version with samples of a copy of the 
original material item, 

generating a recovered code word by comparing the registered copy of the 
origiiial and of the suspect material items, and 
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detecting one or more code words from a correlation between the recovered 
code word and the code words from the set, wherein the associating the samples of the 
suspected and original versions comprises 

forming at least one reduced-bandwidth- version of the 
the original version of the material item, and 

associating the suspected version and the copy of 
accordance with a comparison between the reduced-ban<jlwidth-versions of the 
suspected version and of the original material item 



uspefcted version and of 
the material item in 



12. A detecting data processing apparatus as claimed in Claim 11, wherein 
the forming the at least one reduced-band width- version of the suspected and of the 
original version comprises 

generating a plurality of reduced-bandwidth-versions <bf the suspected and the 
copy of the original material items in accordance with a plurality of levels, each level 
corresponding to an increasing reduction in the bandwidth of the suspected and 
original material items, wherein the association of the suspected version and of the 
copy of the original version is determined in accordance yith a nested alignment 
process. 

13. A detecting data processing apparatus as claimed in Claim 12, wherein 
the association of the suspected version and the original versic n in accordance with the 
nested alignment process comprises 

comparing reduced-bandwidth-versions of the suspected and of the original 
material items according to the lowest level having the smallest bandwidth, 

determining any misalignment between the reduced-b indwidth-versions of the 
suspected and of the original material items according to the lowest level, 

aligning the reduced-bandwidth-versions of the suspected and of the original 
material items according to next lowest level, and 

repeating for the next lowest level until the suspected version and the original 
version of the material item are aligned. 
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14. A computer program providing computer executable instructions, 
whiclji when loaded onto a data processor configures the data processor to operate as a 
detecting data processing apparatus according to any of Claims 1 to 10. 

15. A computer program providing computer executable instructions, 
whicfo when loaded on to a data processor causes the data processor to perform the 
method according to Claim 1 1, 12 or 13. * 

16. A computer program product having a computer readable medium 
10 havinlg recorded thereon information signals representative of the computer program 

claimjed in Claim 14 or 15. 

17. A detecting data processing apparatus as herein before described with 
reference to the accompanying drawings. 



code 



reference to the accompanying drawings. 



18. A method of determining whether at least one of a predetermined set of 
words is present in a suspect material item as herein before described with 
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ABSTRACT 



DATA PROCESSING APPARATUS AND METHOD 



A detecting data processing apparatus determines whether one or more code 

words of a predetermined set of code words are present in i. -suspected version of a 

material item. The apparatus comprises a registration processor operable to associate 

samples of the suspected version with samples of a copy of the original material item, 

a recovery processor and a detection processor. The recovery processor is operable to 

generate a recovered code word by comparing the registered :opy of the original and 

the suspect material items. The detection processor is operable to detect one or more 
» 

code words from a correlation between the recovered code \\ord and the code words 
from the set. The registration processor is operable to form at least one reduced- 
bandwidth-version of the material item and to associate the suspected version and the 
copy of the material item in accordance with a comparison between the reduced- 
bandwidth- versions of the suspected version and the original material item. 

The registration of the suspected version and the 
material item is typically required in order to improve a 
recovering a code word from the suspected version. For material items such as video 
recordings of motion picture films, a substantial number of samples may need to be 
associated in order to effect the registration process. As a result, the registration 
process may be time consuming or at least computationally 
forming reduced-bandwidth- versions of the suspected version 
of the material items, a corresponding reduction in the time taken to perform the 
registration process is provide in accordance with the bandwidth reduction. 
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